Time-Constrained Temporal Logic Control 
of Multi-AfRne Systems 

Ebru Aydin Gol * Calin Belta * 

* Boston University, Boston, MA 02215, USA 
e-mail: {ebru,cbelta}@bu.edu 



(N 

o 

(N 



(N 

in 
o 



> 

m 

00 

in 

en 

o 

(N 



'■% 



Abstract: In this paper, we consider the problem of controlhng a dynamical system such that 
its trajectories satisfy a temporal logic property in a given amount of time. We focus on multi- 
affine systems and specifications given as syntactically co-safe linear temporal logic formulas 
over rectangular regions in the state space. The proposed algorithm is based on the estimation 
of time bounds for facet reachability problems and solving a time optimal reachability problem 
on the product between a weighted transition system and an automaton that enforces the 
satisfaction of the specification. A random optimization algorithm is used to iteratively improve 
the solution. 



1. INTRODUCTION 

Temporal logics and model checking algorithms have been 
primarily used for specifying and verifying correctness of 
software and hardware systems. Due to their expressivity 
and resemblance to natural language, temporal logics 
have gained popularity as specification languages in other 
areas including dynamical systems. Recently, there has 
been increasing interest in formal synthesis of dynamical 
systems, where the goal is to generate a control strategy 
for a dynamical system from a specification given as a 
temporal logic formula, such as Linear Temporal Logic 
(LTL) (Kloetzer and Belta (2008a); Tabuada and Pappas 
(2003); Girard (2010a)), or fragments of LTL, such as 
GR(1) (Gazit et al. (2007); Wongpiromsarn et al. (2009)) 
and syntactically co-safe LTL (Bhatia et al. (2010)). 

We focus on a particular class of nonlinear affine control 
systems, where the drift is a multi- affine vector field {i.e., 
affine in each state component), the control distribution 
is constant, and the control is constrained to a convex 
set. This class of dynamics includes the Euler, Volt err a 
(Volterra (1926)) and Lotka-Volterra (Lotka (1925)) equa- 
tions, attitude and velocity control systems for aircraft 
(Nijmeijer and van der Schaft (1990)) and underwater 
vehicles (Belta (2004)), and models of biochemical net- 
works (de Jong (2002)). In Belta and Habets (2006), the 
authors studied the problem of synthesizing a state feed- 
back controller such that the trajectories originating in a 
rectangle leave it through a specified facet. These results 
were generalized in Habets et al. (2006) by allowing the 
trajectories to leave through a set of exit facets. 

In this paper, we consider the following problem: given a 
multi-affine control system and a syntactically co-safe LTL 
formula over rectangular subregions of the state space, 
find a set of initial states for which there exists a control 
strategy such that all the trajectories of the closed-loop 
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system satisfy the formula within a given time bound. 
Syntactically co-safe LTL formulas can be used to describe 
finite horizon specifications such as target reachability 
with obstacle avoidance: "always avoid obstacle O until 
reaching target T", sequencing constraints "do not go to 
A OY B unless C was visited before", and more complex 
temporal and Boolean logic combinations of these. Our 
approach to this problem consists of two main steps. First, 
we construct a finite abstraction of the system by solving 
facet reachability problems on a rectangular partition of 
the state space. We build on the results from Belta and 
Habets (2006); Habets et al. (2006) to derive bounds for 
the exit times of the trajectories. Second, we solve time 
optimal reachability problems on the product between the 
abstraction and an automaton that enforces the satisfac- 
tion of the specification. We propose an iterative refine- 
ment procedure via a random optimization algorithm. 

Finite abstractions for controlling dynamical systems have 
been widely used, e.g by Tabuada and Pappas (2003). 
Time optimal control of dynamical systems through ab- 
stractions has been studied by Mazo and Tabuada (2011) 
and Girard (2010b). In both cases, an optimal controller is 
synthesized for an approximate abstraction, which is then 
mapped to a suboptimal solution for the original system 
for specifications given in the form of "reach and avoid" 
sets. While our solution also involves an optimal control 
problem on the abstraction, our automata-theoretic ap- 
proach allows for richer, temporal logic control specifica- 
tions. 

The remainder of the paper is organized as follows. We 
review some notions necessary throughout the paper in 
Sec. 2 before formulating the problem and outlining the 
approach in Sec. 3. A review of facet reachability problems 
and the derivation of the exit time bounds are presented 
in Sec. 4. The control strategy providing a solution to 
the main problem is described in Sec. 5 and the random 
optimization method for refinement is given in Sec. 6. An 
example is given in Sec. 7 and conclusions are summarized 
in Sec. 8. 



2. PRELIMINARIES 

2.1 Transition systems and linear temporal logic 

Definition 1. A weighted transition system is a tuple T = 
(Q, S, ^, O, o, It;), where Q and S are sets of states and 
inputs, (^ : Q X E — > 2^ is a transition map, O is a set 
of observations, o : Q — > O is an observation map, and 
w '. Q xY^ — > IR+ is a map that assigns a positive weight 
to each state and input pair. 

5{q^ a) denotes the set of successor states of q under the 
input a. If the cardinahty of 6{q^ a) is one, the transition 
S{q, a) is deterministic. A transition system T is cahed 
deterministic if ah its transitions are deterministic. 

A finite input word ai . . . a^ , cr^ G S, i = 1, . . . , n and an 
initial state qo ^ Q define a trajectory r = q^ . . .q^ of the 
system with the property that g'^+i G (5(^^,(7^+1) for all 
< z < n — 1. The cost J^{r) of trajectory r is defined as 
the sum of the corresponding weights, i.e., 

n-l 

A trajectory r = go • • • <7n produces a word o{qo) . . . o{qn)- 
Definition 2. ( Kupferman and Vardi (2001)) A syntac- 
tically co-safe LTL (scLTL) formula over a set of atomic 
propositions 11 is inductively defined as follows: 

^ := it\^it\^ V <l>|^ A ^^U^J^^, (1) 

where tt G 11 is an atomic proposition, -i (negation), V 
(disjunction), A (conjunction) are Boolean operators, and 
hi ("until"), and J^ ("eventually") are temporal operators 



The semantics of scLTL formulas is defined over infinite 
words over 2^. Informally, 7riZ//7r2 states that tti is true 
until 7r2 is true and 7r2 becomes eventually true in a word; 
J^TTi states that tti becomes true at some position in 
the word. More complex specifications can be defined by 
combing temporal and Boolean operators (see Eqn. (27)). 

An important property of scLTL formulas is that, even 
though they have infinite-time semantics, their satisfaction 
is guaranteed in finite time. Explicitly, for any scLTL 
formula ^ over 11, any satisfying infinite word over 2^ 
contains a satisfying finite prefix. 

Definition 3. A deterministic finite state automaton (FSA) 
is a tuple A = (5, 11, (5^, /So, F) where 5 is a finite set of 
states, n is an input alphabet, ^o ^ 5* is a set of initial 
states, F C S* is a set of final states, and 5j,\ S xli — > S 
is a deterministic transition relation. 

An accepting run r^ of an automaton ^ on a finite 
word w = Wo . . .Wd over E is a sequence of states r^ = 
So . . . Sd-\-i such that so G ^o, Sd-\-i G F and SA{si,Wi) = 
Si-^i for all i = 0, . . . , d. For any scLTL <l> formula over 
n, there exists a FSA A with input alphabet 2^ that 
accepts the prefixes of all the satisfying words. There 
are algorithmic procedures and off-the-shelf tools, such as 



^ The scLTL syntax usually includes a "next" temporal operator. We 
do not use it here because it is irrelevant for the particular semantics 
of continuous trajectories that we define later. 



scheck2 by Latvala (2003), for the construction of such an 
automaton. 

Definition 4- Given a weighted transition system T = 
(g,E,(5,0,o,^) and a FSA A = (6',n, (5^,5'o,F) with 
O = n, their product automaton is a FSA A^ = 
{Sp, Ti^Sp, Spo, Fp) where Sp = Q x S is the set of states, 
S is the input alphabet, 6p : Sp x I] — > 2^^ is the 
transition relation with (5p((g', 5), a) = {{q'^s') \ q' G 
(5((7, cf)., ^^(5, o{q)) = s'}^ Spo = Q x So is the set of initial 
states, and Fp = Q x F is the set of final states. 

An accepting run rp = (<7o7 -^o) • • • (<7n7 -^n) of ^^ de- 
fines an accepting run so . . . s^ of A over input word 
o{qo) . . . o{qn-i) ' The weight function of the transition 
system can directly be used to assign weights to transitions 
of ^^, i.e., we can define a weight function for the product 
automaton in the form wp{Sp{{q^ ^)^^)) — ^{^{q^ ^))- The 
corresponding cost for a run rp = ((70, ^o) . . . {qn^ Sn) of ^^ 
over <Ji . . . (Jn is defined as 

n 

J^{rp) = ^wp{6p{{qi-i,Si-i),ai)). 



2.2 Rectangles and multi-affine functions 

For A^ G N, an A/'-dimensional rectangle RN{cL^b) C M^ 
is characterized by two vectors a = (ai,...,aAr) and 
b = (^i,...,^Ar) with the property that a^ < hi for all 
i = l,...,N: 

RN{a, b) = {x €R^ \\fi € {1, . . . ,N} : Ui < Xi < bi}. 

(2) 

Let V(a, b) and J^(a, b) be the set of vertices and facets of 
of RN{ci^b)^ respectively. Let F^^^ denote the facet with 
normal ±e^, where e^, i = 1,...,A/' denote the standard 
basis of R^. For a facet F G J^(a, 6), V{F) denotes its set 
of vertices and np denotes its outer normal. For a vertex 
V G V(a, 6), Ty denotes the set of facets containing v. 

Definition 5. A multi-affine function h : R^ — > W (with 
A^, g G N) is a function that is affine in each of its variables, 
i.e., h is of the form 



ii,...,iArG{0,l} 



^N •> 



h[xi, . . . ,xn) — / ^ '^«iv,*7v*^i 

ii,...,iArG{0,l} 

with Ci, i^r ^ R^ for all ii, . . . , z^v ^ {0, 1}, and using the 



convention that if i^ 



0, then x]^ 



1. 



Belta and Habets (2006) showed that a multi-affine func- 
tion /i on a rectangle RN{a^b) is uniquely defined by its 
values at the vertices, and inside the rectangle the function 
is a convex combination of its values at the vertices: 

N 

h{xi,...,XN)= ^ W 



aj 



br 



t)eV(a,6) i=l 



(3) 



bi 



h{v). 



where ^i : {a^,6^} — > {0, 1} is an indicator function such 
that (^ilai) = and ^^(6^) = 1 for ah i = 1, . . . , A/". 

3. PROBLEM FORMULATION 

Consider a continuous-time multi-affine control system of 
the form 
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Fig. 1. Examples of continuous trajectories of system (4). 
The atomic propositions are shown in the rectangles 
where they are satisfied. 

x(t) = h{x{t)) + Bu{t), x{t) e Rn{o^, b^),u{t) e U 

(4) 
where i?Ar(a^,6^) C R^ , B e R^^^, and the control 
input u{t) is restricted to a polyhedral set U C M^. 

Rectangular regions of interests in i?Ar(a^, b^) are defined 
using a set of atomic propositions 11 = {iTi \ i = 
0, . . . , /}. Each atomic proposition tt^ is satisfied in a set of 
rectangular subsets of the state space of system (4), which 
is denoted as: 

[tt,] = U^tii^iv(a^'"% ^^'"0 C RN{a^. b^), d, G N. (5) 

The specifications are given as scLTL formulas over the 
set of predicates 11. A trajectory of system (4) satisfies 
the specification if the word produced by the trajectory 
satisfies the corresponding formula. Informally, while a 
trajectory of system (4) evolves, it produces the satisfy- 
ing predicates and the sequence of predicates defines the 
word produced by a trajectory. Specifically, a trajectory 
produces predicate tt^ whenever it spends a finite amount 
of time in a rectangle where tt^ is satisfied. For example, 
trajectories {x{t)}o<^t<^^i and {a;(t)}o<t<r2 shown in Fig. 1 
produce the words 7ri7ro7r37ri7r2 and 7ri7r27ri7ri, respectively. 
The word produced by a trajectory depends on how the 
rectangles are defined. The presented approach employs a 
refinement procedure based on adding hyperplanes, which 
induces smaller rectangles that inherit the predicate. For 
example, if the dashed line in Fig. 1 is added, the tra- 
jectory {x{t)}Q^t^^2 produces 7ri7r27r27ri7ri. As discussed 
by Kloetzer and Belta (2008a), when LTL without next 
operator is considered, 7ri7r27ri7ri and 7ri7r27r27ri7ri satisfy 
the same set of LTL formulas. 

Remark 1. In this paper, we study finite time trajectories 
of system (4). When infinite time trajectories are of 
interest, invariant controllers can be considered as in 
Habets et al. (2006). 

Problem 1. Given a syntactically co-safe LTL formula ^ 
over a set of predicates 11 and a time bound T, find a set 
of initial states Xq C Rn{(^^'^) and a feedback control 
strategy such that all words produced by the closed-loop 
trajectories of system (4) originating in Xq satisfy the 
formula in time less than T. 

Our proposed solution to Prob.l starts with a proposition- 
preserving rectangular partition^ of i?Ar(a^,&^), i.e., 
each element of the partition is a rectangle RN{a,b) C 
^Ar(<^-^'^S ^-^'^0 for some j = 1, . . . , d^, i — 0, . . . , / from 
Eqn. (5). For each rectangle in the partition, and for 
each subset of its set of facets, we derive state-feedback 
controllers driving all the initial states in the rectangle 



through the set of facets in finite time by using the 
sufficient conditions derived in Habets et al. (2006). We 
compute upper bounds for these times and choose the 
feedback controllers that minimize the upper bounds for 
each rectangle and each set of exit facets. We then con- 
struct a weighted transition system, in which the states 
label the rectangles from the partition, the inputs label the 
controllers, and the weights capture the time bounds. We 
find an optimal run of this transition system that satisfies 
the formula by solving an optimal reachability problem 
on its product with an FSA that accepts the language 
satisfying the formula. The rectangles corresponding to the 
initial states with costs less than T compose the set Xq. In 
order to increase this set, we use an iterative refinement of 
the partition based on a random optimization algorithm. 

4. FACET REACHABILITY PROBLEMS 

In this section, we focus on the derivation of the facet 
reachability controllers and their corresponding time 
bounds. We first summarize the sufficient conditions for 
facet reachability from Habets et al. (2006): 
Theorem 1. Let RN{ci^b) be a rectangle and E C F{a^b) 
be a non-empty subset of its facets. There exists a multi- 
affine feedback controller k : Rn{(i^ b) — > U such that all 
the trajectories of the closed-loop system (4) originating 
in RN{ci-, b) leave it through a facet from the set £ in finite 
time if the following conditions are satisfied: 

nl{h{v)^Bk{v))<{),\JF e:F^\£ V^GV(a,6), (6) 
Conv{{h{v) + Bk{v) I V G V(a, b)}) (7) 

where Conv denotes the convex hull. 

In particular, when the cardinality of £^ is 1, i.e. 8 = {F}, 
then Eqns. (6) and (7) imply that the speed towards the 
exit facet F has to be positive everywhere in RN{ci-, b), i.e. 

< nl{h{v) + Bk{v)),\/v e V(a, b). (8) 

As a consequence, for this particular case, the sufficient 
conditions (6) and (7) can be replaced with (6) and (8). 

The linear inequalities given in (6) and (8) (or (6) and 
(7)) define a set of admissible controls Uy for each vertex 
V G V(a, 6). By choosing a control for each vertex v from 
the corresponding set Uy, we can construct a multi-affine 
state feedback controller k that solves the corresponding 
control problem by using Eqn. (3). We first provide a time 
upper bound for the case when there is only one exit facet 
(Prop. 1), and then use this result to provide an upper 
bound for the general case (Cor. 1). 

Proposition 1. Assume that k : RN{ci^b) — > U is an 
admissible multi-affine feedback controller that solves the 
control-to- facet problem for a facet F G J-'(a, b) with outer 
normal e^ of a rectangle i?Ar(a, b). Then all the trajectories 
of the closed loop system starting in rectangle RN{a,b) 
leave the rectangle through facet F in time less than T^, 

where 

bi - ai 



= ln(^)- 

Sf Sf 



Sf 



(9) 



with 



■^ We use the term "partition" loosely in this paper. The rectangle 
boundaries are irrelevant, since due to the synthesized controllers the 
trajectories never slide along the boundaries. 



Sf = min {{h{v) ^ Bk{v))i), 
veviF) 

J^ = min_ {{h{v) + Bk{v))i), 
veviF) 



where F denotes the facet opposite to F, i.e. with normal 



-d. 



Proof: Let x G Rn{(^^ b) and x^, xP be the projections of 
X on F and F, respectively. Then, we have x 



bj—Xj 

hi — ai 



XP^ 



(1 ~ b-V- )^^' -^^^ every x G i?Ar(<^,^), h{x) is a convex 
combination of {h{v) \ v G V(a,6)}. Furthermore, if x 
belongs to a facet of i?Ar(a,6), then h{x) is a convex 
combination of the values of h at the vertices of that facet. 
Therefore, we have 

SF < {h{xP) ^ Bk{xP))i, (10) 

sW < {hC^) ^ Bk(^))i. (11) 

Since k{x) is a solution of the control- to- facet problem for 
facet F, the speed towards F is positive everywhere in 
i?Ar(tt,^), hence 



< s{x) := 



b^ 



Xi , b, 

-^5^+(l 



6. 



'-)SF < {h{x)^Bk{x))i. 

(12) 

For any x G i^Ar(a, 6), the speed in the i^^ direction is lower 
bounded by s{x) (Eqn. (12)), which depends linearly on 
Xi. Since system (13) defined below is always slower than 
the original one, its time upper bound to reach facet F 
gives a valid upper bound for the original system. 



Xi{t) = 



-SF + (1 



Xi 



)sF,Xi G [ai.hi] (13) 



hi - ai hi- ai 

The explicit solution of Eqn. (13) is given in Eqn. (14), 
where x^ denotes the i^^ component of the initial condi- 
tion. 



Xi{t) = exp( 



SFi — sf 
hi - ai 



t){x^i- 



hiSF — CLiSF N biSF — CLiSF 



Sf — Sf ' Sf — sf 

(14) 
Solving (14) for time T^ at x(T^) = hi gives the time 

i i 

upper bound from Eqn. (15). Any trajectory starting from 
an initial point x in i?Ar(a, h) with Xi = x^ reaches the facet 
F^^ in time less than T^ . 



rpF 



ln(- 



h 



iSF — cgsF 



bjSf 



1 Sp—Sp 

As T^o attains its maximum when x? 



Sf — Sf 



(15) 



T^=ln(- 



biSF — ajSF 

Sp—SF 

biSF — ajSF 



Sf 



ai . .Sf. hi- 

-^ =ln(^) 

Sf Sf Sf 



ai 



Sf 



(16) 
gives the upper bound for all x G RN{a, h). □ 

Prop. 1 uses the fact that if k : RN{a^h) — > /7 is a 
solution to the considered control-to-facet problem, then 
the speed n^{h{x) + Bk{x)) towards the exit facet is 
positive for all x G RN{a,h). By defining a slower system 
using minimum speeds on F and F towards the exit facet, 
a time bound for the original system is found. A more 
conservative time bound T'^ can be computed using only 
the minimum speed towards F, i.e. T'^ = _^^-q^__ 

While it is more efficient to compute T'^, T^ gives a 
tighter bound (T^ < T'^). Indeed, the computation of 
T^ considers the change on the lower bound of speed with 
respect to Xi. Moreover, while sf gets closer to s^, T'^ 
approaches T^: 



X2 




Fig. 2. (a) Rectangle [-1.5,-1] x [-0.2,0.2] and sample 
trajectories originating in facet F~^^ . (b) Simulation 
times according to the initial condition in X2 and 
time bounds T^ ^ (red lines) computed using Eqn. 
(9) for controllers synthesized from (19) for e = and 
6 = 0.2. 



T . ^SF.hi-ai hi 
lim ln( — ) ^ = — : 

sf^^sf Sf Sf — Sf 



Sf 



(17) 



Remark 2. The time bound T^ from Eqn. (9) is attainable 
in some cases. Let Vsp^ = argmin^^v(F) {{h{v) -\- Bk{v))i) 
and v^ 



argmin^^^^;^^ {{h{v) + Bk{v))i). If 



{^sf)j = {^sf)j^ 

{h{vs^)^Bk{vs^))j=0, 

{h{v7^) + Bk{v^))j =0, j = 1, . . . , TV, jV ^, 



(18) 



then the trajectory originating at Vsp. G F reaches Vsp G F 
at time T^. 

For each vertex v G V(a,6), we can minimize the time 
bound given in Prop. 1 if we choose a control Uy G Uy that 
maximizes nJ^{h{v)-\-Buy). Computationally, this involves 
solving a linear program at each vertex of a rectangle. 
Formally, at each vertex v^ the optimization problem can 
be written as: 

max n^{h{v) + Buy) 



nl.{h{v)^Buy) < 

Uy eU 



-e,WF' eTy\F 



(19) 



where < e, which is a robustness parameter guaranteeing 
that a trajectory never reaches a facet other than F while 
moving towards F. Decreasing e relaxes the problem (19) 
by increasing the size of the feasible region, which results 
in higher speeds and tighter time bounds. Note that when 
< e the equalities given in Eqn. (18) can not hold, since 
for a vertex v the speed towards a facet F' G J^v\F is upper 
bounded by — e. Therefore the robustness parameter e also 
affects the distance between the time bound from Eqn. (9) 
and the actual maximal amount of time required to reach 
F. 

The tightness of the time bound from Eqn. (9) and 
the effects of the robustness parameter e are illustrated 
through an example in Fig. 2, where the control problem 
for exit facet F^^ of rectangle [-1.5,-1] x [-0.2,0.2] is 
considered for the control system from Eqn. (26). Some 
trajectories of the closed loop system obtained by using 
the feedback controller that minimizes T^ ^ when e = 0.2 
are shown in Fig. 2a. The corresponding times for reaching 
F^^ for e = and e = 0.2 are shown in Fig. 2b. Note 



that when e = 0, the trajectory starting from (—1.5,0.2) 
reaches facet F^^ exactly at time T^ ^ . 

Corollary 1. Given a rectangle Rn{o^^ b) and an admissible 
multi-affine feedback k : i?Ar(a,6) — > U that solves the 
control problem from Thm. 1 with set of exit facets S, 
all trajectories of the closed loop system originating in 
rectangle i?Ar(a,6) leave it through a facet F e E in time 
less than 

T^ =minT^, 

Fes 

where each T^ is computed as in Prop. 1 if < n^{h{v) + 
Bk{v)) for all v G V(a, h). Otherwise T^ is set to oo. 

Proof: hei F e £ with < n^{h{v) + Bk{v)) for ah 
V G V(a,6). Then by Prop. 1 every trajectory originating 
in RN{a,b) reaches F within time T^ (9) unless it leaves 
RN{a^b) before reaching F. Hence, mini?^^: T^ gives a 
valid bound to the control-to-set-of-facets problem for 
£. □ 

For a facet reachability problem with £ as the set of 
exit facets, T^ is computed for each F ^ £ through 
choosing controls that minimize T^ (9) and satisfy the 
linear inequalities defined in Thm. 1. Computationally, 
this translates to solving the following linear program for 
each V G V(a, b) and for each F e £: 

max n^{h{v) + Buy) 

nj, {h{v) -^Buy)< -e, VF' eTy\£ 
Uy eU (20) 

where e is defined as in optimization problem (19). 

As already stated, T^ foiFeE is calculated as in (9) if 
the speeds at all vertices are positive towards F. In this 
case, the condition from (7) is trivially satisfied. Then a 
multi-affine feedback k is constructed by using the controls 
where min^^^^; T^ attains its minimum. 

5. CONTROL STRATEGY 

In this section, we provide a solution to Prob. 1 for a 
proposition-preserving partition of Rn{o?^^ b^). We use the 
results from Sec. 4 to construct a weighted transition 
system from the partition and find an optimal control 
strategy for the weighted transition system. The control 
strategy enforces the satisfaction of the specification and 
maps directly to a strategy for system (4). 

A proposition-preserving partition of RN{a^^b^) and so- 
lutions of facet reachability problems for the rectangles in 
the partition set define a weighted transition system T = 
(Q, E, S, O, o, w). Each state g G Q of T corresponds to a 
rectangle Rn{o.^^ b^) in the partition set. An input a G S of 
T indicates a non-empty subset of the facets of a rectangle 
and a transition 5{q^ a) is introduced if the corresponding 
control problem has a solution. Specifically, we consider a 
facet reachability problem for each state g G Q and each 
non-empty subset of J^(a^,6^), and find the multi-affine 
feedback control which minimizes the corresponding time 
bound as explained in Sec. 4. The successors of S{q^ a) are 
the states q' such that R^^a^^b^) and Rn^cl^ , ^^ ) have 
a common facet in a. The transition weights are assigned 
according to the time bounds computed as described in 



Prop. 1 and Cor. 1. O equals to the set of predicates 11 

and o{q) = iTi if RN{a^^ b^) ^ [Tr^]. 

All words that satisfy the specification formula <I> are 
accepted by a FSA A = iS, 11, (5^, ^o, F) ^ . We construct 
a product automaton A = {Sp^T^^Sp^ Spo^Fp) from T 
and A as described in Def. 4. 

A control strategy (Sq^Q) for A^ is defined as a set of 
initial states Sq and a state feedback control function 
Q : Sp — > S implying that Q{s) will be the input at state 
s. The state feedback function Q characterizes the set of 
initial states Sq C Spo such that every run sqSi . . . Sn of 
A^ starting from a state sq in Sq is an accepting run over 
the word Q{so) . . . ^^(5^-1). Since A^ is non-deterministic, 
there can be multiple runs starting from a state sq G 
Sq under the feedback control Q. In literature (Kloetzer 
and Belta (2008b), Wolfgang (2002)), non-determinism is 
resolved through a reachability game played between a 
protagonist and an adversary, and Sq is defined as the 
set of initial states such that the protagonist always wins 
the game by applying Q. Next, we introduce an algorithm 
based on fixed-point computation to find a maximal Sq 
and corresponding feedback control Q through optimizing 
a cost for each s G Sp. Asarin and Maler (2009) used a 
similar algorithm to solve optimal reachability problems 
on timed game automata. 

Remark 3. Generally, the reachability games are consid- 
ered over an infinite horizon such as Buchi games, where 
winning a game for the protagonist means identifying and 
reaching an invariant set of "good" states. As we consider 
FSAs, the acceptance condition coincides with finite time 
reachability. Hence, a simple reachability algorithm is suf- 
ficient in our case. 

Let Jq : Sp ^ M+ be a cost function with respect to a set 
of final states Fp and feedback control Q such that any run 
of A^ starting from s reaches a state / G Fp under the 
feedback control Q with a cost upper bounded by Jn{s). 
Note that if there exists a run starting from s that can not 
reach Fp, the cost is infinity, Jn{s) — 00. 

The solution of the fixed-point problem given in Eqn. (21) 
gives the optimal cost for each s G S. 

J(s) = min(J(5), min max J{s)-\-wp{Sp{s,o-))) (21) 

(rel^ s'eSp{s,a) 

Algorithm 1 Compute J and Q for A^ = (Sp, E, Sp, Spo,Fp) 
1: J{s) = oo,\/s e Sp 

2: J{f) = o,yfeFp 

3: SC = {s\3a G S and / G Fp such that / G Sp{s, a)} 

4: while S'C / do 

5: SC = SC\ {s}, for some s e SC 

6: if min^gs mayisf^Sp(s,a) J(s') + wp(6p(s, a)) < J(s) then 

7: Q{s) = arg minxes max^/g^^ (5^^) J{s') -\- wp{Sp{s,a)) 

8: J{s) = ina^^,^Sp(sMs)) ^(^') + wp{Sp{s, ^(s))) 

9: SC = SCU {s'\3a e^,se Sp(s', a)} 

10: end if 

11: end while 

Alg. 1 implements the solution for the fixed-point problem 
in Eqn. (21) for the states of A^ and finds the optimal 

^ In the general case, as described in Sec. 2, the input alphabet of 
this automaton is 2^. However, since the words generated by system 
(4) are over 11, it is sufficient to consider 11 as the input alphabet for 
the automaton. 



feedback control Q. A finite state cost, J{s) < oo, and a 
feedback control Q resulted from Alg. 1 means that every 
run starting from s reaches a state / in Fp under the 
feedback control Q with a cost at most J{s). Therefore, 
Sq = {s I J{s) < oo, 5 G 5*0 } is the maximal set of initial 
states of A^ such that under the feedback control Q all 
runs starting from Sq are accepting. Consequently, 

SE = {s\J{s)<T,s&So} (22) 

is the maximal set of initial states such that under the 
feedback control Q cost of a run starting from Sq is upper 
bounded by T. 

If only control-to-facet problems are considered while 
constructing the transition system T, T and the product 
automaton A^ become deterministic. Hence, in this case 
it is sufficient to use a shortest path algorithm to find 
optimum costs and feedback control ft instead of Alg. 1. 

If a multi-affine feedback k solves facet reachability prob- 
lem for the set of exit facets E C J^(a, b) of rectangle 
RN{ci,b), then /c is a solution of the facet reachability 
problem for every superset £' of £ with the same time 
bound T^ by Cor. 1. While constructing ^ of T, a solution 
is searched for every subset of J^(a, 6), hence 

wp{Spis, £')) < wpiSp{s, S)), if£ C £'. (23) 

In line 6 of Alg. 1, cost of a state is updated according 
to the state with maximum cost among a transitions 
successor states, hence Alg. 1 tends to choose the £ with 
minimum cardinality among the sets £' C J-'(a, b) with the 
same transition cost. 

Control Strategy for T: (Kloetzer and Belt a (2008b)) 
We construct a control strategy (Qo^A^) for T using 
the control strategy (S^^Q) for A resulted from Alg. 1 
and Eqn.(22). The set of initial states Qo is the projec- 
tion of Sq to the states of T. Since the feedback con- 
trol Q for A-^ becomes non-stationary when projected 
to the states of T, we construct a feedback control for 
T in the form of a feedback control automaton A^ = 
(5'c, Q^^Ci Scoi Fc^ ^c, S)- The feedback control automa- 
ton A^ reads the current state of T and outputs the input 
to be applied to that state. The set of states 5'c, the set of 
initial states Scq and the set of final states Fc of A^ are 
inherited from A^ the set of inputs Q is the states of T. The 
memory update function Sc ' Sc x Q — > Sc is defined as 
Sc{s^q) = 5j,{q^o{q)) if 5j,{q^o{q)) is defined. The output 
alphabet E is the input alphabet of T. ^c • Sc x Q — > S 
is the output function, Vtci^i q) — ^{{q^ <§)) if J{{q^ <§)) < T 
and Qc{s^q) is undefined otherwise. 

If we set the set of observations of T to Q and define the 
observation map o as an identity map, then the product 
of T and A^ will have same states and transitions as A^ . 
Hence, the words produced by trajectories of T starting 
from Qo in closed loop with A^ satisfy ^. 

Control strategy (Qo^A^) for T is used as a control 
strategy for system (4) by mapping the output of A^ 
to the corresponding multi-affine feedback controller. This 
strategy guarantees that every trajectory of system (4) 
originating in Xq given in Eqn. (24) satisfies ^ in time less 
than T. 

Xo= U ^iv(a^6^) (24) 

qeQo 



For every xq G Xq, there exists an initial state q ^ Qo 
and s G Sco such that xq G RN{ci^^b^) and (q^s) G Sq 
from Eqn. (22). Let /c^ j^^^^ g) be the multi-affine feedback 
which solves control-to-facet (or control-to-set-of-facets) 
problem on R^ia^^ b^) for Q^ci^i q) as the set of exit facets. 
Starting from xq multi-affine feedback ks^Q^^s,q) is applied 
to system (4) until the trajectory reaches a facet F G 
^c{s, q) with a positive speed towards F. By construction 
of A^ ^ it is guaranteed that the trajectory reaches a facet 
F G Vtc{s^q) in time less than w{5{q^Vtc{s^q))). Then 
the applied multi-affine feedback switches to k^/ ^q^(^s' ,q') 

where F = i?Ar(a^,6^) fl i?Ar(a^', 6^') and s' = Sc{s,q). 
This process continues until a final state f e Fc of AP is 
reached. 

Theorem 2. The trajectories of system (4) originating in 
Xq (24) with control strategy {Qq^A^) satisfies $ in time 
less than T. 

Proof: By Def. 4, every word produced by an accepting 
run of ^^ satisfies $. Hence, by construction of {Qq^A^) 
and Xq the words produced by closed loop trajectories 
of system (4) originating in Xq satisfy ^. Consider a 
finite trajectory {x(t)}o<t<r of system (4) with x(0) G 
Xq evolving under the control strategy {Qq^A^). Let 
^c = sq^i ' ' ' Sn be the corresponding run of A^ ^ rj- = 
qoqi ' ' -qn be the corresponding trajectory of T and U be 
a time instant when control switch occurs, i.e. at time 
ti, the trajectory hits a facet F G Qc{si-i,qi-i) with a 
positive speed towards F while evolving under the multi- 
affine feedback A:s._i,^^(s-_i,q._i), for alH = 1, . . . , n and 
tn = r. By Prop. 1 and Cor. 1, for alH = 1, . . . , n: 

U - U-i < w{8{q,_^,^c{s,_i.qr-i))) = T^^(^^-'^^-). 

(25) 
By Alg. 1, r < J((go, ^o)) and by Eqn. (24) r<T. D 

In Thm. 2, we showed that the proposed feedback con- 
trol strategy solves Prob. 1 for a proposition-preserving 
partition of RN{a^^b^). Next we describe an iterative 
refinement procedure to increase the volume of Xq. 



6. REFINEMENT 

An iterative refinement procedure is employed to enlarge 
the set Xq (24). As mentioned before, the rectangles de- 
fined by the set of predicates induce an initial proposition- 
preserving grid partition of RN{ct^^ b^). A grid partition is 
defined by a set of thresholds {d^j^^^ foi" each dimension 
l<j<N. 

Introducing a new threshold di in dimension j can affect 
Xq in different ways and it does not always enlarge the 
set Xq. Consider a state s G Spo with J{s) as computed 
in Alg. 1 and corresponding rectangle i?Ar(a, 6) with aj < 
di < bj. Assume a multi-affine feedback k : RN{a^b) — > 
U solves the control-to-facet problem for a facet F G 
J^(a, b) with outer normal e^ and assume the corresponding 
time bound is T^ as given in Prop. 1. When Rjy^a^b) is 
partitioned into two rectangles i?Ar(«,^*) and i?Ar(<^*,^) 
through a hyperplane Xj = di, we need to consider two 
cases: j = i and j 7^ i, which are illustrated in Fig. 3 on a 
rectangle in M?. 



F 



F" 



F** 



F* 



F" 
F' 



R2(a,h) R2{a,b*) R2{a\h) i?2(a,6*) 

Fig. 3. Two partitioning schemes for R2{a.b) C M?. 

(a){i = j} Since state feedback k solves the control-to- 
facet problem on i?Ar(a,6) for F, the speed towards the 
exit facet is positive for all x G RN{cL^b). Moreover, no 
trajectory leaves i?Ar(«,^) through another facet. Hence, 
k solves the control-to- facet problems on i?Ar(a,6*) and 
i^Ar(«*, b) for the facets with normal e^. Let T^ and T^ 
be the corresponding time bounds. Then when k is applied, 
any trajectory starting in i?Ar(a, 6*) and i?Ar(<^*, b) reaches 
F within time T^ + T^ , which is upper bounded by 
T^ . The proof follows from the proof of the Prop. 1, the 
minimal speed towards F on the intersection of RN{a^b) 

and Xi = di is lower bounded by ^,_^, ~sf + ^*_^' gF- As 

the actual minimal speed could be higher than ^^^, ~sf + 

,*~^' gF and other multi-affine feedbacks could solve the 

bi — ai ^ 

same problem on i^Ar(a, 6*) and i?Ar(<^*, b) with lower time 
bounds, when i = j, partitioning results in tighter time 
bounds. 

(b){i y^ j} The multi-affine feedback k solves the control- 
to-set-of- facets problem on i?Ar(a,6*) for exit facets S^ = 
{F^,F*} where npf = e^ and np* = Cj. Moreover, k 
solves control-to-set-of- facets problem on i?Ar(a*,6) for 
exit facets S'^ = {F'\F**} where n^'' = e^ and np** = 
—Cj. Then the corresponding time bounds T^ and T^ 
are upper bounded by T^ by Cor. 1. However, T^ or T^ 
could be higher than T^, hence, the costs of the resulting 
automaton states could be higher than J{s). 

In (a) and (b)^ the effects of partitioning are analyzed on 
a rectangular region for a simple case where the initial 
rectangle has a solution to the control-to-facet problem 
for facet F. It is concluded that when a rectangle Rn^cl-, b) 
of a state s e Sp with J{s) is partitioned, the costs of the 
resulting states s' and s'^ can be higher or lower than J{s). 
Hence, even for that simple case, partitioning can have 
negative and positive effects on the defined time bound 
for a single rectangle. Moreover, there is no closed form 
relationship between the partitioning scheme {dl}i^f^ and 
the volume of the set Xq. 

In order to overcome these difficulties, we use a Particle 
Swarm Optimization (PSO)(Trelea (2003)) algorithm to 
find the new thresholds. The objective of the optimization 
is maximizing the volume of the set Xq (24). We run the 
PSO algorithm iteratively. At each iteration, a new thresh- 
old dj^ is added between two consecutive ones dj^dj_^^ 
depending on the distance between them and the value of 
the corresponding optimization variable. An optimization 
variable for dj^ is defined with range [dj , dl_^-^ — d] if the 
distance between two consecutive thresholds is twice as 
large as the minimum allowed edge size, 2d < dj_^-^ — dj. 

Part of the range [dj ^d^ ^ d) is used to decide whether to 
add the threshold or not, i.e. a new threshold is added only 
if dl^ G [dl +(i, d^j^-^ —d] . The dimension of the optimization 
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Fig. 4. (a) FSA A that accepts the language satisfying 
$ (27) (T stands for Boolean constant true). The 
initial state of the automaton is filled with grey and 
the final state is marked with a double circle, (b) The 
initial partition induced by the predicate set H. ttq, 
TTi, 7r2, TTs and 7r4 are satisfied in cyan, magenta, red, 
green and orange colored rectangles, respectively and 
TTs is satisfied in white rectangles. 

problem depends on the grid configuration {<i:^}iGN of the 
iteration. The iterative procedure terminates when either 
all the intervals are smaller than 2d or there is no change 
in the optimum objective value for the last two iterations. 

Remark 4' Let d^ = |{(i:^}^^i^|, then the cardinality of 
the resulting partition is I\.^^i{d^ — 1). Construction of 
the transition system T (see Sec. 5) from the partition 
{dl}ien requires to solve (2^^ — l)HJ^]^((i-^ — 1) linear 
programs. For each partition, in addition to solving these 
linear programs, we take the product between T and A^ 
and run Alg. 1 to find the volume of the set Xq. 

7. CASE STUDY 



(26) 



Consider the following multi-affine system 

Xi = —Xi + X1X2 + u 
±2 = -X2 +X1X2 +ii, 

where the state x and the control input u are constrained 
to sets RN{a^,b^) = [-2,2] x [-2,2] and U = [-1,1], 
respectively. The specification is to visit one of the rect- 
angles that satisfy tti or tts, then a rectangle where ttq is 
satisfied, while always avoiding the rectangles that satisfy 
7r2. Moreover, if a trajectory visits a rectangle where 7r4 
is satisfied, then it has to visit a rectangle that satisfies 
TTs before visiting a rectangle that satisfies ttq. Predicates 
TT^, i = 0, ...,4 are defined in Fig. 4b. Formally, this 
specification translates to the following scLTL formula ^ 
over H = {7ro,7ri,7r2,7r3,7r4,7r5}: 

^ = ((^7r4^/7ro) V {^TToUlTs)) A {^7T2U7To) A (^7ro^/(7ri V TTs)) 

(27) 

A FSA A that accepts the language satisfying formula 
^ is given in Fig. 4a. The regions of interests and the 
corresponding partition are given in Fig. 4b. The upper 
time bound to satisfy the specification is set to T = 2.5, the 
minimum edge length is set to d = 0.2 and the robustness 
parameter for optimization problems (19) and (20) is set 
to e = 0.2. 

To illustrate the main results of the paper, we use two 
approaches to generate a control strategy. In the first 
experiment, only control-to-facet problems are considered, 
hence a deterministic transition system is used. As dis- 
cussed in the paper, the resulting product automaton is 



also deterministic and it is sufficient to use a shortest 
path algorithm instead of Alg. 1. In the second approach, 
both control-to-facet and control-to-set-of-facets problems 
are considered. Hence, the resulting transition system and 
product automaton are non-deterministic, and Alg. 1 is 
applied. 

We use {Q^,A^) and (Qo'^.A^^) to denote the control 
strategies as defined in Sec. 5 for the partition schemes 
resulted from the iterative refinement described in Sec. 6 
for the first and second approach, respectively. We use Xq 
and Xq '^ to denote the corresponding sets of initial states of 
system(26), respectively. These sets, together with sample 
trajectories of the closed loop systems, are shown in Fig. 5. 
The volume of Xq is 5.25 and the volume of Xq^ is 
7.62. A control-to-facet problem on a rectangle i?2(<^, b) C 
[—2,-0.2] X [-2,-0.2] does not have a solution for facets 
F^^ and F^^ because of the strong drift in that region. 
However, rectangles in the same region have solutions 
to control-to-set-of-facets problem for S = {F^^^F^'^}. 
Consequently, rectangles in that region is only covered by 
Xq ^ as the construction of Xq ^ considers non-determinism. 
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Fig. 5. The yellow regions in (a) and (b) represent Xq 
and Xq^, respectively. Some simulated satisfying tra- 
jectories of the corresponding closed-loop systems are 
shown (the initial states are marked by circles). 

8. CONCLUSION 

We studied a time-constrained control problem for a 
continuous-time multi-affine system from a specification 
given as a syntactically co-safe LTL formula over a set of 
predicates in its state variables. Our approach was based 
on finding an optimal control strategy on the product 
between an abstraction of the system and an automa- 
ton enforcing the satisfaction of the specification. The 
abstraction was a weighted transition system constructed 
by solving facet reachability problems on a rectangular 
partition of the state space of the original system. We 
proposed an iterative refinement procedure via a random 
optimization algorithm to increase the set of admissible 
initial states. 
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